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Ms.  Petra  Alfred  (ARL-HRED) 

Dr.  Nita  Miller  and  Dr.  Lawrence  Shattuck,  COL(Ret)  (NPS) 

MORSS  WG26  Presentation 
12  June  2007 


Agenda 


Purpose  of  this  briefing 

To  describe  TRAC-MTRY  survey  development  and  analysis  support  to  the 

LW/MW  DOTMLPF  Assessment  study. 

*  Objectives. 

*  Project  team  and  background  research. 

*  Methodology. 

*  Survey  plan  development. 

*  Survey  instrument  deveiopment  and  administration. 

*  Data  consoiidation  and  anaiysis. 

*  Conclusions. 


LW:  Land  Warrior  MW:  Mounted  Warrior  DOTMLPF:  Doctrine,  Organization,  Training,  Materiel,  Leader  Development,  Personnel,  Facilities 
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Consolidated  Survey  Effort  Objectives 


*  To  design  and  administer  questionnaires  and  focus 
group  interviews  for  large  Soidier  populations  based 
upon  requiring  agency  input. 

*  To  provide  initial  analyses  of  the  questionnaire 
responses  and  focus  group  interviews,  with  additionai, 
more-thorough,  analyses  as  needed. 

*  Advance  the  state-of-the-art  in  survey  development  and 
administration  within  TRAC. 

-  Develop  an  overarching  methodology. 

-  Develop  supporting  tools. 

-  Demonstrate  effective  application  of  advanced  analysis 
techniques. 
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Project  Team 


Survey  effort  project  team. 

-  MAJ  Eric  Tollefson,  lead  analyst 
and  project  lead. 

-  MAJ  Jon  Alt,  supporting  analyst. 

-  LTC  Jeffrey  Schamburg,  senior 
analyst. 

-  Mr.  Kevin  Wainer,  TRAC-WSMR. 

-  Mr.  Eric  Johnson,  TRAC-MRO. 

-  Dr.  Nita  Miller,  Naval 
Postgraduate  School  (NPS), 
Operations  Research  (Human 
Systems  Integration). 

-  Dr.  Lawrence  Shattuck,  COL 
(Ret),  NPS,  Operations  Research 
(Human  Systems  Integration). 

-  Dr.  Lyn  Whitaker,  NPS, 
Operations  Research  (Statistics). 

-  Ms.  Petra  Alfred,  NPS  thesis 
student.  Army  Research  Lab  - 
Human  Research  and 
Engineering  Directorate  (ARL- 
HRED). 


Other  contributing  agencies. 

-  TRAC-WSMR,  analysis  lead. 

-  MAJ  Pedro  Habic. 

-  Mr.  Eddie  Edwards. 

-  CPT  Clark  Adams. 

-  TRADOC  Capabilities  Manager 
(TCM)-Soldier,  study  lead. 

-  Army  Research  Institute  (ARI). 

-  Army  Evaluation  Center  (AEC). 

-  US  Army  Armor  Center. 

-  US  Army  Infantry  Center 
Directorate  of  Training  (DOT). 

-  ARL-HRED. 

-  US  Army  Materiel  Systems 
Analysis  Activity  (AMSAA). 

-  TRAC-LEE. 

-  Army  Capabilities  Integration 
Center  (ARCIC). 

-  Office  of  Infantry  Proponency 
(OIP). 

-  Project  Manager  -  Soldier 
Warrior  (PM-SWAR). 
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Background  Research 


•  Previous  TRAC  documentation  concerning  survey 
development. 

-  Cioppa,  Thomas  and  Chris  Morey,  TRAC,  “A  Primer  for 
Conducting  Surveys,  Questionnaires,  and  Assessments,”  dated 
12  May  05. 

-  Daniel,  John,  “Survey  Design  and  Analysis:  A  Course  Presented 
for  the  U.S.  Army,  Ft.  Leavenworth.” 

-  Anderson,  Mike,  TRAC,  “Survey  Definitions.” 

•  Project  team  expertise. 

-  Drs.  Miller  and  Shattuck  have  extensive  experience  in  survey 
administration  and  analysis,  and  teach  it  in  their  graduate-level 
courses. 

-  Many  of  the  other  team  participants  also  have  extensive  survey 
expertise,  particularly  ARI  and  AEC. 
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Solicit  and  Analyze  SME  /  Stakeholder  Input 


•  Required  to  integrate  question  input  from  numerous  contributing 
agencies. 

•  Developed  a  question  input  spreadsheet  that  requested  the 
following  elements  of  information  for  each  question. 

-  Study  issue,  essential  element  of  analysis  (EEA),  measure  of 
effectiveness  (MOE)  being  addressed. 

-  Recommended  question. 

-  Priority  compared  to  other  submitted  questions. 

-  Recommended  format  of  the  question  (Likert,  interview,  etc.). 

-  Recommended  respondent  (all,  leader  only,  etc.). 

-  Rationale  for  question  inclusion. 

-  Analysis  requirements. 


Unique  question  input  spreadsheet  provided  traceability  from  questions 
to  study  issues,  which  was  essential  for  survey  development. 
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Develop,  Refine,  and  Pilot  Questions 


•  Integrated  input  from  contributing  agencies. 

-  Selected  questions  for  inclusion. 

-  Based  upon  question  priority. 

-  Combined  similar  questions  to  remove  redundancy. 

-  Adjusted  question  format. 

-  Developed  questions  to  fill  perceived  gaps. 

•  Developed  draft  questionnaires  and  interview  questions. 

•  Sent  drafts  to  contributing  agencies  for  revision. 

•  Piloted  draft  questionnaires  with  groups  of  Soldiers  at  the  Defense 
Language  Institute  (DLI). 

•  Printed  4  questionnaire  versions:  LW  Leader,  LW  Non-leader,  MW 
Vehicle  Commander  (VC),  and  MW  Driver. 


Question  development  was  an  iterative  process  that  integrated 
stakeholder  input  and  pilot  group  feedback  within  a  continuous 

development  cycle. 
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Administer  Questionnaires  and  Interviews 


•  Questionnaires. 

-  Longest  version  of  the 
questionnaires  (LW  Leader) 
took  approximateiy  1.5  hrs  to 
compiete. 

*  Focus  group  interviews. 

-  Four  per  company. 

-  LW  junior  ieader  (squad 
ieader  and  beiow). 

-  LW  senior  ieader  (piatoon 
sergeant  and  above). 

-  LW  non-ieader. 

-  MW  vehicie  crewman. 

-  Totai  of  16  focus  groups,  each 
consisting  of  5-7  Soidiers. 

-  interview  team  inciuded  an 
experienced  faciiitator  and  a 
recorder. 

-  Each  interview  iasted 
approximateiy  1  hour. 


Questionnaires  Compieted 


Date 

16 

Oct 

17 

Oct 

18 

Oct 

20 

Oct 

All 

Unit 

BCo 

A  Co 

CCo 

HHC/ 

EN 

Pit 

LW 

Leader 

37 

43 

26 

12 

118 

LW 

Non- 

Leader 

38 

35 

37 

10 

120 

MW 

Vehicle 

Cdr 

18 

9 

8 

5 

40 

MW 

Driver 

17 

11 

12 

4 

44 

Questionnaires  and  focus  group  interviews  effectiveiy  captured  the 
opinions  of  a  iarge  portion  of  the  4-9  iN  popuiation. 
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Questionnaire  Demographics  Summary 


Demographic 

Overall 

Land  Warrior 

Mounted  Warrior 

Overall 

Leader 

Non-Ldr 

Overall 

VC 

Driver 

Number  of  respondents 

322 

238 

103 

135 

84 

38 

46 

Average  age 

24.2 

24.2 

26.9 

22.1 

24.1 

26.0 

22.6 

Average  time  in  duty 
position  (months) 

9.8 

9.6 

9.6 

9.6 

10.2 

9.9 

10.5 

Average  years  of  service 

3.7 

3.8 

6.1 

2.0 

3.3 

5.7 

2.0 

Average  years  of  active 
service 

3.3 

3.4 

5.2 

1.9 

3.1 

4.4 

1.9 

%  of  Soldiers  with 
deployment  experience 

28% 

33% 

61% 

23% 

14% 

29% 

2% 

Average  deployment  time 
(of  Soldiers  that  deployed, 
in  months) 

15.9 

15.9 

16.7 

10.9 

16.1 

16.5 

13 
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Consolidate  Data 


•  Questionnaires. 

-  Manually  entered  the  data  into  the  database. 

-  Analyzed  data  for  detectable  errors  (invalid  values,  blanks, 
unusual  patterns). 

-  Estimated  error  rate. 

-  Randomly  chose  10  questionnaires. 

-  Checked  every  entry  for  errors. 

-  Found  14  errors  in  5,045  data  points  for  an  error  rate  of  about  0.3%. 

•  Focus  group  interviews. 

-  Recorded  interviews  digitally. 

-  Recorders  compiled  final  interview  summaries  from  notes  taken 
during  the  interviews  and  the  digital  recordings. 


Unique  techniques  were  used  to  mitigate  and  characterize  data  entry  error. 


Note-taking,  in  conjunction  with  digital  recordings,  was  an  ideal  combination 
of  data  collection  methods  for  the  focus  group  interviews. 
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Analyze  LW  Basis  of  Issue  (BOI)  Responses 


•  Received  a  short  notice  requirement  to  provide  initiai  BOI  resuits  by 
25  Oct  06. 

•  Data  entry. 

-  Focused  on  seiected  BOI  questions  and  iimited  demographic 
sections  of  the  238  compieted  LW  questionnaires. 

-  Entered  112,220  data  points  in  2.5  days. 

•  Initial  data  summary. 

-  Focus:  desired  LW  functions  by  duty  position. 

-  Conducted  item-level  summaries  for  key  questions. 

-  Resuits  reported  by  totai  popuiation  and  by  ieader  versus  non¬ 
leader. 

•  Subsequent  data  summaries. 

-  Provided  results  by  enlisted  versus  officer/NCO  and  by 
company  on  01  Nov  06. 

-  Coior-coded  the  spreadsheets  to  reveai  patterns. 

Consolidated  survey  team  was  able  to  enter  BOI  data  and  provide 
useful  first-cut  analyses  in  a  very  short  period  of  time. 
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Example  BOI  Results 


15.  Vhich  of  the  following  positions  should  have  each  ot  the  following  Land  Vairior  ? 


RifI 

Enginec 

Engineer  Platoon  \ 

Engineer  Squad  Leader 

EngineerTeam  Leader 

Reconnaissance  fScouH  Platoon  Leader 


Reconnaissance  [ScoutI  Platoon  Sergeant 
Reconnaissance  (Scout]  Squad  Leader] 


Reconnaissance  [Scoml  Team  Leader 


Rifle  Platoon  Leader 

Pprrpntanp  ^79%^  of  rII 

Rifle  Platoon  Sergeant 

Rifle  Squad  Leader 

responaenis  wno  leit 

Rifle  Team  Leader 

that  tho  Riflo 

a.  View  situation  awareness  displa|; 

(1)  Self-locate  (own  icon  or  GBI)  onlj. 

180X 

82X 

82X 

77X 

82% 

i-L;cfUL;i  ^iiuuiu  iicivl;  iiil; 

(2)  Select  other  individuals  to  displa|. 

84X 

92X 

83X 

SOX 

82% 

abilitv  to  self-locate. 

b.  Manipulate  map  displai. 

74% 

74X 

84X 

87X 

SOX 

82% 

c.  Use  measure  distance  tool. 

SIX 

87X 

70X 

S3X 

87X 

71X 

70X 

SSX 

S4X 

S7X 

d.  diqital  : 

(1)  Call  for  fire. 

53X 

S4X 

78X 

78X 

SOX 

SIX 

74X 

SSX 

SOX 

SSX 

(2)  Call  for  medic. 

77X 

78x 

SSX 

SOX 

77X 

79X 

78X 

77X 

75X 

78X 

(3)  SALUTE. 

64X 

74X 

SOX 

78X 

73X 

71X 

74X 

73X 

87X 

70X 

(4)  Free  teit. 

6IX 

87X 

78X 

78X 

S3X 

SSX 

74X 

73X 

87X 

70X 

e.  /iecfive  diqital  mfsssees. 

55X 

SSX 

70X 

S9X 

S7X 

SSX 

74X 

73X 

S7X 

70X 

(.  Use  DVS  to  detect  and  engage  targets. 

18X 

18X 

14X 

13X 

27X 

29X 

22X 

I8X 

2IX 

I7X 

g.  Use  HMD  to  view  TVS  images. 

22X 

24X 

22X 

22X 

30X 

29X 

2SX 

23X 

2IX 

I7X 

h.  Use  HMD  to  viewlsend  target  information  from  the  MFLISTORM. 

30X 

43X 

3SX 

42X 

SOX 

SOX 

S2X 

SSx 

SSX 

48X 

i.  fteceive/Kfif  diqital  d/ei/iMS, 

72X 

78X 

84X 

82X 

70X 

71X 

74X 

73X 

7SX 

74X 

j.  ffecfivf/yffr  digital  cftiefs  . 

SOX 

SSX 

78X 

78X 

S3X 

84X 

70X 

SSX 

SSX 

SSX 

k.  Vfftf  preloaded  digital  im^pes. 

SSX 

74X 

78X 

SOX 

70X 

7SX 

74X 

73X 

SSX 

70X 

1.  S'sve 

digital  imapes. 

55X 

S4X 

70X 

71X 

S7X 

79X 

74X 

73X 

S7X 

SSX 

m.  Sena  digital  unifies. 

55X 

S2X 

SSX 

S7X 

S7X 

7SX 

74X 

73X 

SSX 

70X 

n.  Use  voice  communications. 

SSX 

83X 

84X 

82X 

77X 

82X 

78X 

82X 

7SX 

74X 

o.  digital 

34X 

54X 

SSX 

SOX 

73X 

7SX 

74X 

73X 

SOX 

SSX 

p.  digital 

3IX 

SIX 

70X 

82X 

S3X 

7SX 

70x 

73X 

SOX 

SSX 

q.  Crente  digital  c-rtfers. 

20X 

3SX 

SSX 

71X 

37X 

48X 

SIX 

SSX 

29X 

43X 

r.  MeOift  diqital  erraefs. 

2IX 

34X 

SSX 

78X 

37X 

48X 

S7X 

SSX 

29X 

43X 

Total  Respondents  Vho  Chose  to  Evaluate  the  Dutt  Position 

34 

7S 

SO 

4S 

30 

28 

23 

22 

24 

23 

Adjustable  color-coding  of  BOI  responses  provides  the  analyst  a 
means  to  detect  patterns  in  responses  across  duty  positions 

and  across  capabiiities. 
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Analyze  Questionnaire  Responses 


•  Provided  data  summaries  to  the  Assessment  team. 

-  Totai  number  of  respondents  and  number  who  chose  each  possibie 
response  ievei. 

-  Percentage  of  respondents  who  chose  each  potentiai  response. 

-  Percentage  of  respondents  who  chose  each  potentiai  response,  except 
“N/A”  or  non-entries  (“Percent  of  Those  with  Opinion”  beiow). 

-  Broken  down  by:  ieader/non-ieader,  officer&NCO/eniisted,  company. 

•  No  means,  standard  deviations,  etc. 

-  Responses  were  ordinai  in  nature. 

-  Response  ievei  intervais  cannot  be  considered  equai  or  constant. 


OVE 

f^ALL 

Number 

Question 

Sample  Size 

Much  Worse 

Worse 

About  the 

Same 

Better 

Much  Better 

N/A;  No  Basis 

to  Judge 

No  Entry 

1A1 

Situational  Awareness.  Compared  to 
your  standard  equipment,  Land  Warrior 
makes  your  ability  to  monitor  the 
location  of  your  unit  members... 

Number  Responded 

238 

7 

24 

68 

85 

40 

12 

2 

Percent  of  Total 
Responded 

238 

2.9% 

10.1% 

28.6% 

35.7% 

16.8% 

5.0% 

0.8% 

Percent  of  Those 
with  Opinion 

224 

3.1% 

10.7% 

30.4% 

37.9% 

17.9% 

Provided  summary  anaiyses  in  an  easy-to-read  format  for  aii  questions 
whiie  avoiding  the  common  pitfails  in  categoricai  data  anaiysis. 
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Analyze  Association  between  Sub-populations 


•  By  question. 

-  Explored  association  between  the  responses  of  certain  sub¬ 
populations  (leader  vs.  non-leader,  A/C  vs.  B  Company*). 

-  Considered  both  original  response  levels  and  combined 
response  levels  (e.g.,  combining  “Strongly  Disagree”  and 
“Disagree”  and  combining  “Agree”  and  “Strongly  Agree”). 

-  Tested  for  statistical  significance  for  each  question. 

•  Question  groupings. 

-  Explored  associations  between  the  sub-populations  of  interest 
across  groupings  of  questions. 

-  Used  a  combination  of  statistical  techniques  to  detect  evidence 
of  association  across  the  question  groupings. 

•  B  Company  had  a  different  BOI  and  participated  in  unique  training  events  as  part  of 

the  US  Army  Test  and  Evaiuation  Command  (ATEC)  Limited  User  Test  (LUT). 

These  results  allow  the  analyst  to  determine  the  value  of  examining  the 
resuits  by  sub-popuiation  versus  the  overaii  popuiation  and  to  see  trends 

across  question  groupings. 
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Association  Analysis  Results 

by  Question 


•  Provided  results  for: 

-  All  questions. 

-  Original  and  adjusted 
response  levels. 

-  Three  statistical  tests.* 

•  Color-coded  the  levels  of 
significance. 

•  Identified  questions  for  which 
required  statistical 
assumptions  are  violated. 

•  Provided  a  methodology  for 
determining  significance  based 
upon  the  results  of  the  three 
statistical  tests. 

^Likelihood  ratio  (LR)  chi-square  test, 

Fisher’s  exact  test  (only  for  2  by  2  tables), 
Cochran-Mantel-Haenszel  test 

(for  A/C  vs  B  after  blocking  for  leader  vs 
non-leader). 


Example  of  the  Association 
Results  Spreadsheet 


Results  for  3 

statistical 

tests. 


Assumptions 
violated  for 
statistical 
test. 


Color-coded 

significance 

ievels. 
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Association  Analysis  Results 

by  Question  Groupings 


Overall  Sub-population  Association  for  Land  Warrior  Questionnaires 


Section 

Leader  versus  Non-Leader 

A/C  Co  versus  B  Co 

Capability  Gaps 

Significant 

Significant 

BO1 1-12 

Significant 

Significant 

BO1 13-14 

Not  Significant 

Significant 

BOI  15 

N/A* 

Not  Significant 

Training 

Significant 

Not  Significant 

Other  Implications 

Significant 

Significant 

*BOI  Question  15  was  not  asked  of  non-leaders. 


Overall  Sub-population  Association  for  Mounted  Warrior  Questionnaires 


Section 

VC  versus  Driver 

A/C  Co  versus  B  Co 

Capability  Gaps 

Significant 

Significant 

Training 

Significant** 

Significant 

Other  Implications 

Not  Significant 

Not  Significant 

**Not  significant  if  Fisher’s  exact  test  p-values  are  not  included  when  available. 


For  each  question  grouping,  the  overall  sub-population  association  either  tends  toward 

“Significant”  or  “Not  Significant”. 
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Conclusions 


•  Refined  and  improved  survey  methodologies  and  capabilities 
through  the: 

-  Development  of  a  generalized  survey  development  and  analysis 
methodology. 

-  Development  of  supporting  tools. 

-  Effective  application  of  advanced  analysis  techniques. 

•  Keys  to  success. 

-  Wide  range  of  stakeholder  input  for  the  creation  of  a  holistic 
survey. 

-  Use  of  multiple  analysis  techniques  to  mitigate  risk  and  clearly 
identify  most  important  aspects  of  the  system. 

•  Significant  effort. 

-  Input  from  12  different  Army  and  DoD  agencies. 

-  322  completed  questionnaires  consisting  of  over  189,000  data 
points. 

-  16  hours  of  focus  group  interviews. 

•  Application  of  these  techniques  provided  a  significant  contribution 
to  the  overarching  study  results,  particularly  with  respect  to 
capability  gaps,  training,  and  leader  development  issues. 
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Using  Unique  Questionnaire  and  Interview 
Techniques  to  Inform  Acquisition 

Decision-Making 


MAJ  Eric  Tollefson,  MAJ  Jon  Alt,  and  LTC  Jeffrey  Schamburg  (TRAC-MTRY) 

Ms.  Petra  Alfred  (ARL-HRED) 

Dr.  Nita  Miller  and  Dr.  Lawrence  Shattuck,  COL(Ret)  (NPS) 

MORSS  WG26  Presentation 
12  June  2007 


Milestones 


Activity 

Completed 

Data  requirements  sent  to  TRAC  in  Monterey  (TRAC-MTRY)  by  requiring  agencies. 

17  JUL06 

Initial  drafts  developed  and  sent  out  for  agency  input  (DRAFT  #1). 

28  JUL  06 

Feedback  on  initial  drafts  provided  to  TRAC. 

10  AUG  06 

Revised  drafts  developed  and  sent  out  for  agency  input,  to  include  draft  focus 
group  interview  questions  (DRAFT  #2). 

11  AUG  06 

Feedback  on  revised  drafts  provided  to  TRAC. 

22  AUG  06 

Pilot  questionnaires  and  focus  group  interviews  executed. 

30-31  AUG  06 

Initial  Institutional  Review  Board  (IRB)  /  Human  Use  Committee  (HUC)  approval 
received. 

05  SEP  06 

Revised  drafts  developed  (DRAFT  #3);  meeting  with  analysis  lead  (TRAC-WSMR) 
conducted. 

06-07  SEP  06 

Revised  drafts  developed;  sent  out  for  final  agency  input  (DRAFT  #4). 

21  SEP  06 

Questionnaires  complete;  sent  to  printer. 

29  SEP  06 

Final  NPS  IRB/HUC  approval  received. 

13  OCT  06 

Questionnaires  and  focus  group  interviews  conducted. 

16-20  OCT  06 

Initial  LW  BOI  analysis  sent  to  TRAC-WSMR. 

25  OCT  06 

Refined  LW  BOI  analysis  complete. 

01  NOV  06 

Sent  LW  and  MW  questionnaire  response  databases  to  requiring  agencies. 

20  NOV  06 

Initial  analysis  of  all  questionnaire  responses  complete. 

18  DEC  06 

Final  analysis  of  all  questionnaire  responses  complete. 

08  FEB  07 
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Agency  Contributions 


*  ARI:  demographics,  BOI,  training,  ieader  deveiopment. 

*  AEC:  force  effectiveness/capabiiity  gaps,  miscelianeous. 

*  ARL-HRED:  human  factors. 

*  TRAC-LEE:  iogistics. 

*  TCM-Soldier  (Ft.  Knox):  Mounted  Warrior. 

*  TRAC-WSMR:  capabiiity  gaps,  BOi,  organization, 
personnel,  faciiities. 

*  AMSAA:  communications. 
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Mounted  Warrior  Land  Warrior 


Questionnaire  Demographics  Summary 


Rank 

MOS 

Education  Levei 

Duty  Position 

Prof  Degree 

Some  Grad 

Bachelors 

_  23 

3-4Yr  College 

9 

Tech  School  ^ 

Assoc 

3 

1-2Yr  College 

53 

HS 

13' 

Some  HS  c 

COCDR 

3 

CO  xo 

2 

1SGT 

2 

CO  FS  NCO 

3 

Rifle  PL  _ 

8 

Rifle  PSG 

8 

FO 

1 

5 

Medic 

]9 

RTO 

6 

Rifle  SL 

|32 

Rifle  TL 

[30 

MGTm 

132 

Rifle  Tm  Mem 

|73 

VDrv  _ 

Recon  PL/PSG 

2 

Recon  SL/TL  _ 

Sniper  _ 

2 

Scout  _ 

1 

EN  PL/PSG  _ 

2 

EN  SL/TL  _ 

Eng  _ 

] 

5 

other 

1 

Some  Grad 

1 

Bachelors 

J4 

3-4Yr  College 

_  3 

Tech  School 

3 

Assoc 

2 

1-2Yr  College 

20 

HS 

51 

VC 

38 

Driver 

46 
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Analytical  Model  for  Association  between  Variables 


Leader  Role 

Soldier 

(Leader  vs 

Responses 

Non-Leader) 

Assigned 

Company 


Unit 

BOI 


Deployment 

Experience 


Training  ^ 


Independent  Variable  Dependent  Variable 


Relationship  Explored  Directly 
Relationship  Explored  Indirectly 


Focused  on  those  variables  (or  sub-populations)  for  which  the  variable 
was  expected  to  have  the  greatest  effect  -  leadership  role  and  company. 
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Association  by  Question  Groupings 


*  Interested  in  seeing  if  the  associations  between  the  sub¬ 
populations  (independent  variables)  of  interest  could  be 
extended  to  groupings  of  questions. 

*  Used  the  observation  that  if  there  is  oniy  a  random 
association  within  a  group  of  questions,  their  p-vaiues 
shouid  be  randomly  distributed  as  a  uniform  (0,1) 
random  variabie. 

*  Tests:  for  each  group  of  questions  we  compared  the 
observed  p-values  against  a  uniform  (0,1)  distribution 
using  a  combination  of: 

-  The  Kolmogorov-Smirnov  (K-S)  test. 

-  Mean  Cl  test:  A  comparison  of  the  calculated  confidence 
interval  (Cl)  for  the  mean  of  the  observed  values  against 
the  expected  value  of  a  uniform  (0,1)  random  variable 
(0.5). 

-  If  the  Cl  is  entirely  below  0.5:  tends  toward  significance. 

-  If  the  Cl  overlaps  0.5:  inconclusive. 

-  If  the  Cl  is  entirely  above  0.5:  tends  toward  non-significance. 
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Association  Analysis  Results  by  Question  Groupings 


OVERALL  SUB-POPULATION  ASSOCIATION  FOR  LAND  WARRIOR  QUESTIONNAIRES 

Section 

Leader  versus  Non-Leader 

A/C  Co  versus  B  Co 

Combined  Levels 

Original  Levels 

Combined  Levels 

Original  Levels 

Capability  Gaps 

Significant 

Significant 

Significant 

Significant 

BOI  1-12 

Significant 

Significant 

Significant 

Significant 

BOI  13-14 

Not  Significant 

Not  Significant 

Significant 

Significant 

BOI  15 

N/A 

N/A 

Not  Significant 

Not  Significant 

Training 

Significant 

Significant 

Not  Significant 

Not  Significant 

Other  Implications 

Significant 

Significant 

Significant 

Significant 

OVERALL  SUB-POPULATION  ASSOCIATION  FOR  MOUNTED  WARRIOR  QUESTIONNAIRES 

Section 

VC  versus  Driver 

A/C  Co  versus  B  Co 

Combined  Levels 

Original  Levels 

Combined  Levels 

Original  Levels 

Capability  Gaps 

Significant 

Significant 

Significant 

Significant 

Training 

Significant 

Not  Significant 

Significant 

Significant 

Other  Implications 

Not  Significant 

Not  Significant 

Not  Significant 

Not  Significant 

For  each  question  grouping,  the  overall  sub-population  association  either  tends  toward 

“Significant”  or  “Not  Significant”. 
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K-S  Test  Results  by  Question  Groups 


KOLMOGOROV-SMIRNOV  TEST  RESULTS  FOR  LAND  WARRIOR  QUESTIONNAIRES 

Section 

Leader  versus  Non-Leader 

A/C  Co  versus  B  Co 

Combined  Levels 

Original  Levels 

Combined  Levels 

Original  Levels 

Capability  Gaps 

Significant 

Significant 

Significant 

Significant 

BOI  1-12 

Significant 

Significant 

Significant 

Significant 

BOI  13-14 

Significant* 

Significant* 

Significant 

Significant 

BOI  15 

N/A 

N/A 

Not  Significant 

Not  Significant 

Training 

Significant 

Significant 

Not  Significant 

Not  Significant 

Other  Implications 

Significant 

Significant 

Significant 

Significant 

KOLMOGOROV-SMIRNOV  TEST  RESULTS  FOR  MOUNTED  WARRIOR  QUESTIONNAIRES 

Section 

VC  versus  Driver 

A/C  Co  versus  B  Co 

Combined  Levels 

Original  Levels 

Combined  Levels 

Original  Levels 

Capability  Gaps 

Significant 

Significant 

Significant 

Significant 

Training 

Significant 

Not  Significant 

Significant 

Significant 

Other  Implications 

Not  Significant 

Not  Significant 

Not  Significant 

Not  Significant 

*Not  significant  if  Fisher’s  exact  test  p-values  are  not  included. 


For  each  question  grouping,  the  K-S  test  either  indicates  that  the  p-values  appear  to  be 
randomly-distributed  (“Not  Significant”)  or  not  (“Significant”). 
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Mean  Cl  Test  Results  by  Question  Groups 


MEAN  Cl  TEST  RESULTS  FOR  LAND  WARRIOR  QUESTIONNAIRES 

Section 

Leader  versus  Non-Leader 

A/C  Co  versus  B  Co 

Combined  Levels 

Original  Levels 

Combined  Levels 

Original  Levels 

Capability  Gaps 

Significant 

Significant 

Significant 

Significant 

BOI  1-12 

Significant 

Significant 

Significant 

Significant 

BO1 13-14 

Not  Significant* 

Not  Significant* 

Significant 

Significant 

BOI  15 

N/A 

N/A 

Inconclusive 

Inconclusive 

Training 

Significant 

Significant 

Not  Significant 

Inconclusive 

Other  Implications 

Significant 

Significant 

Significant 

Significant 

MEAN  Cl  TEST  RESULTS  FOR  MOUNTED  WARRIOR  QUESTIONNAIRES 

Section 

VC  versus  Driver 

A/C  Co  versus  B  Co 

Combined  Levels 

Original  Levels 

Combined  Levels 

Original  Levels 

Capability  Gaps 

Significant 

Significant 

Significant 

Significant 

Training 

Significant 

Significant 

Significant 

Significant 

Other  Implications 

Inconclusive 

Inconclusive 

Inconclusive 

Inconclusive 

^’’Inconclusive”  if  Fisher’s  exact  test  p-values  are  not  included. 

For  each  question  grouping,  the  mean  Cl  test  either  indicates  that  the  p-values  tend 
toward  significance  (“Significant”),  toward  non-significance  (“Not  Significant”),  or 

inconclusive  (“Inconclusive”). 
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